1. Field of the Invention
The present invention relates to a scheme for GUI (graphical user interface) using a polygonal-line-shaped slider that serves as an interface for enabling a user to intuitively manipulate and playback spatio-temporal media data such as video data and animation data continuously in time in correspondence to user input with respect to interactive manipulation and playback of the spatio-temporal media data.
2. Description of the Background Art
One of conventional interfaces for interactively manipulating data is a slider. FIG. 1 shows one conventional slider. The slider comprises a segment having a start point 91 and an end point 93, and a thumb 92, where the thumb 92 moves along the segment. A user uses a pointing device such as a mouse or a touch panel to manipulate the thumb 92. A value between the start point 91 and the end point 93 is set in accordance with the movement of the thumb 92. The value corresponding to the position of the thumb 92 is entered into a personal computer. In FIG. 1, the thumb 92 can be moved between a value 0 assigned to the start point 91 and a value 100 assigned to the end point 93 and is presently positioned at a value 45.
The slider is usually displayed at a lower part of a monitor screen and serves as a cursor (anchor) moving along a time axis. The slider is used to indicate the relative position of a currently playbacked image frame and functions as an interface for realizing variable-speed playback of video data such as random playback, fast forward, and reverse mode. A technique of employing a slider to interactively manipulate data is disclosed in xe2x80x9cA Magnifier Tool for Video Dataxe2x80x9d by M. Mills et al., CHI ""92 (Human factors in computing systems), pp. 93 to 98, 1992.
The conventional slider may continuously manipulate image frames of video data according to the movements of the thumb. This conventional slider, however, linearly projects the time axis of video data onto the slider regardless of the contents of the video data such as the movements of an object contained in the video data. Therefore, the manipulation of the slider does not always agree with playbacked video data intuitively. For example, an object in playbacked video data may move to the left although the thumb of the slider is moved to the right. The conventional slider is unable to continuously follow the complicated trajectory of moving objects, waveforms, or routes in maps while inputting continuous values, because the conventional slider is incapable of spatially relating the video data to the slider, so that video data manipulation is difficult.
Another conventional technique for interactive manipulation of video data is disclosed in U.S. patent application Ser. No. 09/099,189. This technique employs a function for mapping input X-Y coordinates to frame numbers of the images with regard to an object moving along a trajectory that can be linearly mapped. The function is used to playback a frame image corresponding to input coordinates. This technique, however, is also unable to manipulate video data that involve an complicated trajectory.
An object of the present invention is to provide a scheme for GUI (Graphical User Interface) using a polygonal-line-shaped slider that enables a user to intuitively manipulate and playback spatio-temporal media data such as video data and animation data without deteriorating the temporal continuity of the data in interactive manipulation and playback of the spatio-temporal media data.
Another object of the present invention is to resolve the difficulty of manipulation due to the polygonal line shape of the slider, and to improve the handling of the polygonal-line-shaped slider by properly adjusting the position of the thumb of the slider to input coordinates even if user is incapable of tracing the polygonal line or the thumb of the polygonal-line-shaped slider.
In order to accomplish the above objects, an aspect of the present invention provides a method of entering data corresponding to specified coordinates. The method includes the steps of (a) storing coordinates of vertexes which define a polygonal line shaped slider that is composed of a polygonal line formed by at least one segment and a coordinates indicator to be moved along the polygonal line, (b) storing input coordinates, and (c) mapping the input coordinates onto a point on the polygonal line and positioning the coordinate indicator at the point.
Another aspect of the present invention provides a method of interactively manipulating video data. The method includes the steps of (l) sliding a coordinate indicator along a polygonal line formed by at least one segment using a polygonal line shaped slider composed of the polygonal line and the coordinate indicator according to input coordinates, and (m) presenting the video data of an image frame corresponding to the position of the slid coordinates indicator.
Still another aspect of the present invention provides an apparatus for entering data corresponding to specified coordinates. The apparatus has (a) a unit for storing coordinates of vertexes which define a polygonal line shaped slider that is composed of a polygonal line formed by at least one segment and a coordinates indicator to be moved along the polygonal line, (b) a unit for storing input coordinates, and (c) a unit for mapping the input coordinates onto a point on the polygonal line and positioning the coordinate indicator at the point.
Still another aspect of the present invention provides an apparatus for interactively manipulating video data. The apparatus has (l) a unit for receiving input coordinates, (m) a unit for sliding a coordinate indicator along a polygonal line formed by at least one segment using a polygonal line shaped slider composed of the polygonal line and the coordinate indicator according to the input coordinates, and (n) a unit for presenting the video data of an image frame corresponding to the position of the slid coordinate indicator.
Still another aspect of the present invention provides a computer readable recording medium for causing the computer to execute processing for entering data corresponding to specified coordinates, the processing including: (a) a process for storing coordinates of vertexes which define a polygonal line shaped slider that is composed of a polygonal line formed by at least one segment and a coordinate indicator to be moved along the polygonal line, (b) a process for storing Input coordinates, and (c) a process for mapping the input coordinates onto a point on the polygonal line and positioning the coordinate indicator at the point.
Still another aspect of the present invention provides a computer readable recording medium for causing the computer to execute processing for interactively manipulating video data, the processing including: (l) a process for sliding a coordinate indicator along a polygonal line formed by at least one segment using a polygonal line shaped slider composed of the polygonal line and the coordinate indicator according to input coordinates and (m) a process for presenting the video data of an image frame corresponding to the position of the slid coordinate indicator.
Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings.
FIG. 1 is a diagram view showing a conventional slider;
FIG. 2 is a diagram showing a polygonal-line-shaped slider according to the present invention;
FIG. 3 is a block diagram showing a polygonal-line-shaped slider apparatus according to the first embodiment of the present invention;
FIG. 4 is a diagram showing exemplary vertex data stored in a vertex storage unit of the apparatus according to the first embodiment;
FIG. 5 is a flow chart showing a processing carried out by the apparatus according to the first embodiment;
FIG. 6 Is a block diagram showing a detailed configuration of a mapping unit of the apparatus according to the first embodiment;
FIG. 7 is a flow chart showing a processing carried out in step S30 of FIG. 5 according to the first embodiment;
FIG. 8 is a diagram showing a technique for finding a ratio according to the first embodiment:
FIG. 9 is a diagram showing a technique for finding the coordinates of a perpendicular foot, an interior division ratio, and a distance according to the first embodiment;
FIGS. 10 and 11 are diagrams showing relationships between input coordinates and a thumb according to the first embodiment;
FIG. 12 is a block diagram showing a detailed configuration of a mapping unit according to the second embodiment of the present invention;
FIG. 13 is a flow chart showing a processing carried out in step S30 of FIG. 5 according to the second embodiment;
FIG. 14 is a diagram showing a technique for calculating an interior division ratio for a segment according to the second embodiment;
FIG. 15 is a diagram showing a technique for generating a perpendicular line and a bisector line according to the second embodiment;
FIG. 16 is a diagram showing a technique of judging whether or not two points are on the same side of a given line;
FIGS. 17A and 17B are diagrams showing relationships between input coordinates and a thumb according to the second embodiment;
FIGS. 18A and 18B are diagrams showing ranges of input coordinates to move a thumb;
FIG. 19 is a block diagram showing a detailed configuration of a mapping unit according to the third embodiment of the present invention;
FIG. 20 is a flow chart showing a processing carried out in step S30 of FIG. 5 according to the third embodiment;
FIG. 21 is a diagram showing a technique for calculating an interior division ratio for a segment according to the third embodiment;
FIGS. 22A and 22B are diagrams showing a technique for calculating a parallel line of a given segment and an intersection point of input coordinates and a thumb according to the third embodiment;
FIG. 23 is a diagram showing a relationship between input coordinates and a thumb according to the third embodiment;
FIG. 24 is a block diagram showing a polygonal-line-shaped slider apparatus according to the fourth embodiment of the present invention;
FIG. 25 is a flow chart showing a processing carried out by the apparatus of FIG. 24 according to the fourth embodiment;
FIG. 26 is a diagram showing an exemplary polygonal-line-shaped slider provided by an output unit according to the fourth embodiment;
FIG. 27 is a block diagram showing a polygonal-line-shaped slider apparatus according to the fifth embodiment of the present invention;
FIG. 28 is a diagram showing vertex data stored in a vertex storage unit according to the fifth embodiment;
FIG. 29 is a flow chart showing a processing carried out by the apparatus according to the fifth embodiment;
FIG. 30 is a diagram showing an exemplary output provided by an interior division value output unit according to the fifth embodiment;
FIG. 31 is a block diagram showing a polygonal-line-shaped slider apparatus according to the sixth embodiment of the present invention;
FIG. 32 is a flow chart showing a processing carried out by the apparatus according to the sixth embodiment;
FIGS. 33A and 33B are diagrams showing an exemplary case of polygonal line selection provided by a polygonal line selection unit according to the sixth embodiment;
FIG. 34 is a diagram showing the movement of a thumb according to the sixth embodiment;
FIG. 35 is a block diagram showing a polygonal-line-shaped slider apparatus according to the seventh embodiment of the present invention;
FIG. 36 is a flow chart showing a processing carried out by the apparatus according to the seventh embodiment;
FIG. 37 is a diagram showing vertex data stored in a vertex storage unit of the apparatus according to the seventh embodiment;
FIG. 38 is a diagram showing an exemplary case of merging two sliders according to the eighth embodiment of the present invention;
FIG. 39 is a flow chart showing a processing carried out by an apparatus according to the eighth embodiment;
FIG. 40 is a block diagram showing a polygonal-line-shaped slider apparatus according to the ninth embodiment of the present invention;
FIG. 41 is a flow chart showing a processing carried out by the apparatus according to the ninth embodiment;
FIG. 42 is a diagram showing a slider editing provided by an editing unit according to the ninth embodiment;
FIG. 43 is a diagram showing an exemplary case of a polygonal line generated by a polygonal line generation unit according to the tenth embodiment of the present invention;
FIG. 44 is a block diagram showing a polygonal line shaped slider apparatus according to the tenth embodiment;
FIGS. 45A and 45B are diagrams showing exemplary cases of the vertex dividing carried out by a polygonal line correction unit according to the tenth embodiment;
FIG. 46 is a flow chart showing a vertex dividing processing carried out by polygonal line correction unit according to the tenth embodiment;
FIGS. 47A and 47B are diagrams showing the vertex dividing carried out by polygonal line correction unit according to the tenth embodiment;
FIG. 48 is a flow chart showing a vertex merging processing carried out by a modification of polygonal line correction unit according to the tenth embodiment;
FIGS. 49A and 49B are diagrams showing an exemplary case of vertex merging carried out by polygonal line correction unit according to the tenth embodiment:
FIGS. 50A and 50B are diagrams showing one exemplary case of video playback control using the polygonal-line-shaped slider of the present invention;
FIGS. 51A and 51B are diagrams showing another exemplary case of video playback control using the polygonal-line-shaped slider of the present invention;
FIG. 52 is a diagram showing another exemplary case of video playback control using the polygonal-line-shaped slider of the present invention;
FIG. 53 is a diagram showing an exemplary case of map information guidance using the polygonal-line-shaped slider of the present invention;
FIGS. 54A and 54B are diagrams showing an exemplary case of animation data playback using the polygonal-line-shaped slider of the present invention; and
FIGS. 55A and 55B are diagrams showing hardware structures for realizing the polygonal-line-shaped slider of the present invention.